A new indirect encoding method with variable length gene code to optimize neural network structures

A new encoding method for optimizing neural network structures is proposed. It is based on an indirect encoding method with variable length gene code. The search ability for finding an optimal solution is higher than the direct encoding methods because redundant information in gene code is reduced and the search space is also reduced. The proposed method easily operates adding and deleting hidden units. The performance of the proposed method is evaluated through computer simulations. Introduction In designing neural networks (NNs), it is crucial to optimize network structures to derive both optimal approximation ability and generation ability Genetic algorithm (GA) [1, 2] has been successfully applied to variety kinds of optimization problems, such as traveling salesman problem and scheduling problem. In neural computing, there are also a lot of work on designing NNs using GA. One of the most important points is how to encode a network structure and parameters of NN (phenotype) in gene code (genotype), i.e. coding scheme. Dasgupta et al. proposed a design algorithm using structured GA based on a direct encoding method [3], where the network structure and the parameters are directory encoded in gene strings and the optimal solutions of both are searched by GA. On the other hand, Kitano proposed NGL (Neurogenetic Learning) [4]. NGL is based on an indirect encoding method and has interesting properties. In the present paper, a new encoding method for optimizing neural network structures is proposed. It is based on the indirect encoding method with variable length gene code. The search ability for finding an optimal solution is higher than than the direct encoding methods because redundant information in gene code is reduced and the search space is also reduced. It is also possible to easily add or delete hidden units. The performance of the proposed method is evaluated through computer simulations. Encoding Method A coding scheme is shown in Fig.1. This can be applicable to three-layered NNs. input layer output layer bias unit